XML Full-Text Search: Challenges and Opportunities

An ever growing number of XML repositories are being made available for search. A lot of activity has been deployed in the past few years to query such repositories. In particular, full-text querying of text-rich XML documents has generated a wealth of issues that are being addressed by both the database (DB) and information retrieval (IR) communities. The DB community has traditionally focused on developing query languages and efficient evaluation algorithms for highly structured data. In contrast, the IR community has focused on searching unstructured data, and has developed various techniques for ranking query results and evaluating their effectiveness. Fortunately, recent trends in DB and IR research demonstrate a growing interest in adopting IR techniques in DBs and vice versa [1, 2, 3, 4, 5, 6, 7, 9]. In the past 5 years, the W3C has been putting a lot of effort in designing the XQuery 1.0 and XPath 2.0 languages that provide powerful primitives to navigate in XML documents. Many database researchers and practitioners have influenced the design of these languages and have been developing XQuery prototypes. On the other hand, in IR, INEX, the INitiative for the Evaluation of XML [8] has been created 3 years ago to put together XML documents to assess scoring and ranking methods for XML that accounts for document structure, in the same manner as TREC was designed for keyword retrieval. Several prototypes participate to INEX each year and the basic query language used within this effort is very similar to XPath. The goal of this proposal is to provide a survey on existing research in XML full-text search in DB and IR including languages, appropriate scoring and ranking methods, implementation architectures and query evaluation algorithms and, summarize open research issues such as the joint optimization of queries on both structure and content. We believe that this tutorial is necessary to drive the atten-